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Abstract 

Articles on student learning outcomes assessment often treat faculty as 
one homogenous body. Yet the exponential growth of contingent faculty 
in universities and colleges has created two distinct faculty groups with 
varied concerns and thoughts on everything from the future of higher 
education to shared governance to student learning outcomes. When 
considering faculty thoughts and concerns regarding the assessment of 
student learning outcomes, it is inappropriate to assume the concerns of 
tenure-line faculty will echo the concerns of non-tenure-line faculty. In 
this article, we explore survey comments given by non-tenure-line faculty, 
examining the thoughts and concerns of non-tenure-line faculty in regards 
to the creation, implementation, and ramifications of outcomes assessment. 
We find that the contingent status of these faculty creates unique concerns 
that should be considered and addressed by departments and institutions 
wishing to increase participation in outcomes assessment. 


The Other Half: Non-Tenure Track 
Faculty Thoughts on Student Learning 
Outcomes Assessment 


M he literature is growing on faculty involvement in student learning outcomes 
assessment. A number of scholars have discussed the concerns tenure-line faculty have 
with assessment (Fort, 2011; Lederman, 2010a; Lederman, 2010b; Struck, 2007) while 
others explore concerns about the lack of faculty involvement in assessment (Havens, 
2013; Hutchings, 2010; Kuh & Ikenberry, 2009). Assessment clearly divides accreditors, 
administrators, and tenure-line faculty, and this division is reflected in the spate of 
assessment articles in the Chronicle of Higher Education and Inside Higher Ed . Yet a clear 
blind spot has emerged in the assessment literature: The larger discussion mostly ignores 
concerns that non-tenure track (NTT) faculty have with assessment. 
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Understanding the concerns NTT faculty have about assessment is vital to the 
student success movement. At four-year institutions, NTT faculty are more likely to teach 
introductory classes than those on the tenure track. When students walk into a First- 
Year Composition course or Introduction to Philosophy course, the odds are high that 
they will be greeted and taught by an NTT faculty member. A study by the Coalition on 
the Academic Workforce revealed that NTT faculty teach nearly 40% of the introductory 
humanities courses at post-secondary institutions (Modern Language Association, 2001), 
and the American Association of University Professors recently noted that 75.6% of faculty 
positions are not tenure-line (Curtis & Thornton, 2013). The importance of NTT faculty to 
any assessment effort should be obvious: Efforts to improve teaching quality and student 
success must include those teaching a growing percentage of general education classes. 

In a previous article (Scott & Danley-Scott, in press), we discussed ways in which 
officials at two- and four-year universities were communicating assessment goals to their 
NTT faculty. Drawing on results from a 2012 survey of NTT faculty, we argued that efforts to 
offer paid assessment training to contingent faculty might increase participation in sample- 
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collection and ratings stages of outcomes assessment, but, more importantly, bringing NTT 
faculty into the department through communication, recognition, and mentoring, might also 
help. In this study, we found that feeling appreciated by a department made it more likely that 
a NTT faculty member would participate in the assessment activities. 

For the previous article, we did not analyze the respondents’ written comments 
to see how that qualitative data might further enrich or complicate our understanding 
of the relationship between contingent faculty and assessment. We also did not compare 
what respondents reported hearing formally and informally about assessment and student 
learning outcomes. In this paper, we look at these materials to determine whether the 
additional data support our hypothesis that communication and inclusion will increase 
assessment participation. 


Background 


The larger discussion 
mostly ignores concerns 
that non-tenure track 
(NTT) faculty have 
with assessment. 


Although a growing body of literature concerns the pay and work-conditions of adjunct 
faculty, scholarship on student learning assessment still largely ignores NTT faculty and their 
perspectives. This may be due to reticence among institutions and departments to admit the 
extent to which they depend on part-time and adjunct faculty to teach core courses. Outside 
of academia, most are unaware that a two-track hiring and quality control process exists for 
faculty. Tenure-track job candidates are carefully interviewed and carted about campuses to 
give job talks and perform guest lectures, while NTT faculty may be hired at the last minute, 
just weeks before the term starts (June, 2012; Kezar, 2012; Street, Maisto, Merves, & Rhoades, 
2012). And, while not scientific, a perusal of the comments in June’s (2012) Chronicle of 
Higher Education article reveals adjunct faculty sharing stories of being hired with no notice, 
no interview, and no teaching demonstration. 


The omission of NTT faculty from the dialogue on assessment may also be due to 
the feeling by administrators and departments that NTT faculty are of such low quality that 
assessing their classes may prove embarrassing or that teaching workshops will not be attended. 
A growing area of study centers on the teaching quality of faculty off the tenure line, echoing 
these concerns. Some scholars have reported that NTT faculty can negatively affect graduation 
rates (Ehrenberg & Zhang, 2004; Jaeger & Eagan, 2009) and transfer rates (Eagan & Jaeger, 
2009), that they can negatively affect retention (Jaeger & Eagan, 2011), and that they do not 
properly prepare students for courses later in a discipline’s sequence (Burgess & Samuels, 
1999). Bettinger and Long (2004) determined that adjunct faculty can reduce future interest 
in a discipline, though this effect varies by discipline and is minor. Finally, Jacoby (2006) found 
that community colleges with lower part-time faculty ratios tended to have higher graduation 
rates. These findings are summed up in a piece by Benjamin (2002), who argues that more 
tenure-track faculty need to be teaching in undergraduate classrooms because “over-reliance 
[on NTT faculty] particularly disadvantages the less-well-prepared entering and lower-division 
students in the non-elite institutions who most need more substantial faculty attention” (p. 4). 


NTT faculty can, if given 
support and commit¬ 
ment, have a positive 
effect on students. 


As Benjamin’s (2002) proposal is unrealistic, other scholars have explored why NTT 
faculty appear less effective than their tenure-line counterparts. According to this body of 
research, institutional realities rather than inherent qualities may be causing these deficiencies 
in teaching. First, the effectiveness of many NTT faculty may be compromised by institutional 
and departmental policies in staffing (Kezar, 2012). NTT faculty are often thrown into teaching 
assignments with little support or commitment from employing institutions (Umbach, 2007). 
Such faculty are often given little advanced notice of teaching assignments, receive sample 
syllabi only a few weeks before the semester, and are not compensated for preparation or 
office hours (Street et al., 2012). They are less likely to be familiar with student resources 
and institutional opportunities (Green, 2007) that are important for incoming first-year and 
transfer students. 


The above conclusions are supported by recent studies showing that NTT faculty can, 
if given support and commitment, have a positive effect on students. In a study of doctoral 
institutions, Jaeger and Eagan (2011) found a positive correlation between NTT faculty and 
student retention when a doctoral institution provided the support and training for all faculty. 
More recently, a study at Northwestern University caused a stir when it found that 
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a non-tenure track faculty member increases the likelihood that a student 
will take another class in the subject by 7.3 percentage points (9.3 percentage 
points when limited to classes outside the student’s intended major) and 
increases the grade earned in that subsequent class by slightly more than one- 
tenth of a grade point (with a somewhat greater impact for classes outside of 
the intended major). (Figlio, Schapiro, & Soter, 2013, p. 10) 

Discussions surrounding the above study have revealed that Northwestern’s non-tenure track 
faculty are contracted lecturers who are paid comparatively well and do not have to split time 
between research and teaching; they focus on students and teaching quality (Weissmann, 
2013). The Northwestern lesson echoes earlier findings by Baldwin and Wawrzynski (2011) 
that contingent faculty teaching full-time are more likely to have effective teaching practices 
than contingent faculty teaching part-time, as well as the findings of Bettinger and Long 
(2010) that professional adjuncts from technical industries (engineering, business, etc.) 
have specializations that may increase interest and result in students taking future classes 
within majors. 

A related question is whether NTT faculty are as invested in their teaching quality 
as their tenure-line counterparts. When given the opportunity and incentive, do NTT faculty 
participate in training and departmental activities? Again, the literature is varied. Some reports 
indicate contingent faculty in departments are uninvolved (Schmidt, 2013; Umbach, 2007) 
and that part-time faculty are not very responsive to online departmental discussions (Danley- 
Scott & Tompsett-Makin, 2012), while others suggest the lack of participation is not by choice. 
Baldwin and Chronister (2001) observe that contingent faculty are rarely included in higher 
education professional development efforts and governance. Their book features interviews 
with many contingent faculty who wanted to be involved but were turned away. Levin and 
Shaker (2011) note that NTT faculty they interviewed saw themselves as effective teachers and 
believed their teaching effectiveness was important, supporting arguments by Kezar and Sam 
(2011) that NTT faculty should be viewed as freelance professionals who are concerned about 
the quality of their work instead of as piecemeal labor. The latter three studies support and 
explain findings by Scott and Danley-Scott (in press) that NTT faculty are interested in student 
success: Adjuncts surveyed indicated they often took advantage of training at their institutions 
and were generally willing to perform unpaid outcomes assessments if it helped them learn 
more about their teaching effectiveness. And if some institutions really are placing adjuncts 
on governance committees in an effort to meaningfully involve adjuncts in the assessment 
movement (Havens 2013), administrators should witness increasing NTT faculty desire to be 
involved in teaching improvement. 

The above literature implies similarities between the two groups of faculty in terms of 
concern for student success, and we believe those similarities are real. Nevertheless, although 
some might hypothesize that NTT faculty concerns about assessment would also mirror 
those of tenure-line faculty, we question this second assumption. Scholars discussing generic 
“faculty” perceptions of assessment often address two criticisms: that assessment does not 
provide useful information (Hutchings, 2010; Lederman, 2010b) and that assessment is an 
additional obligation without stipends or relief from other duties (Funk & Klomparens, 2006; 
Gilbert, 2010; Gold, Rhoades, Smith, & Kuh, 2011; Havens, 2013). That is, depictions of 
generic faculty attitudes describe assessment as an “unfunded mandate” with little benefit. 
NTT faculty, however, have a tenuous employment situation that already assumes completion 
of unpaid labor and insufficient materials to do a job properly (June, 2012; Kezar, 2012; Street 
et al., 2012). As we will discuss below, regular day-to-day obligations are often performed for 
free by the NTT faculty as a facet of their commitment to teaching, so collecting assessment 
samples and participating in ratings may just be another item on the list rather than something 
to be particularly indignant about. Many NTT faculty also express interest in teaching 
effectiveness (Scott & Danley-Scott, in press), so they may view assessment positively if the 
assessment is truly designed to collect information that provides insight into student learning 
and teaching effectiveness. 

Because we believe the limited communication and contact between NTT faculty and 
departments will increase apprehension about the assessment process, we examine previously 
unreported responses to our 2012 survey of NTT faculty teaching in California institutions. 


If some institutions 
really are placing 
adjuncts on governance 
committees in an effort 
to meaningfully involve 
adjuncts in the assess¬ 
ment movement, admin¬ 
istrators should witness 
increasing NTT faculty 
desire to be involved in 
teaching improvement. 


Many NTT faculty also 
express interest in teach¬ 
ing effectiveness, so they 
may view assessment 
positively if the assess¬ 
ment is truly designed 
to collect information 
that provides insight into 
student learning and 
teaching effectiveness. 
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Most of the assessment 
process happens outside 
of the classroom, which 
leads us to one of the 
potential differences 
between faculty. 


Department chairs may 
be unwilling to ask NTT 
faculty to participate 
in unpaid meetings or 
work, even if it relates 
to assessment, because 
the chair does not want 
to ask a person to work 
for free. 
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These answers and comments to questions on communication channels and departmental 
assessment activity allow us to gauge the validity of our hypothesis. For example, if NTT 
faculty are less likely than tenure-line faculty to attend departmental meetings (Chronicle 
Reporting, 2009) and they are not mentored by tenure-line faculty (Scott & Danley-Scott, 
in press), they will be left in the dark about the design of assessment devices and collection 
of student materials. The resulting ignorance may understandably lead NTT respondents to 
express increased concerns about how departments and institutions will use the results from 
the rating or scoring of the assessment samples and data. 

This is not to say that tenure-line faculty do not worry about the use of data as well, 
but they are more protected from the results than are adjuncts, who lack tenure and are far 
easier to replace due to a more streamlined hiring process for contingent faculty. Moreover, 
units that eliminate a tenure-track faculty member run the risk the line will not be retained, a 
risk not every unit is willing to brave. This shielding sometimes steers tenure-track discussion 
towards other concerns, like academic freedom and methodology. 

Method 

To get a sense of the concerns of NTT faculty, we examined a number of questions from 
our 2012 survey of non-tenure-track faculty teaching at community colleges and universities 
in California. The anonymous, online survey was designed and administered on PsychData; 
the survey link and informed consent were posted on university and college discussion 
boards, campus and union email lists, and social email lists. Faculty were invited to forward 
the information to colleagues and friends. We estimate the original pool of recipients to be 
around 500 potential respondents. The resulting pool of respondents is difficult to estimate, 
as faculty forwarded the survey information to colleagues at colleges and universities outside 
of our initial contact. It is known, through email contact with respondents and initial contact 
points, that faculty from at least nine distinct campuses participated. The survey generated 
70 respondents and 67 usable sets of answers, though not every respondent answered every 
question. Readers interested in viewing the survey may contact the authors. 

Description of the Survey Questions 

In our exploratory survey, we asked faculty about their experiences with student 
learning outcomes assessment. Specifically, we were interested in their training in, feelings 
about, involvement with, and knowledge of student learning outcomes assessment. Questions 
differentiated official messages and experiences with assessment from personal feelings and 
informal messages about assessment. Additional questions were asked to determine the 
effects of work expectations, mentoring opportunities, pay and compensation, professional 
involvements, and general experiences as a faculty member of the institution. In most cases, 
particularly demographic and experience-related questions, respondents were asked to pick 
the response that most closely matched their situation. Some questions in the automated 
survey allowed respondents to select multiple answers, as was the case with the question, 
“What have you heard informally about SLOs?” For a number of questions, respondents had 
the option of selecting “Other” and giving an open response. 

Because numbers only offer part of the picture and can be interpreted beyond what the 
respondents intended, we wanted to evaluate the written comments left by our respondents 
and compare their comments to our interpretations of the numeric results. Two parts of the 
survey provided opportunities for open comment on assessment and the administration of 
assessments. Thirty of the 67 respondents used the open response opportunities to share 
experiences and concerns. Although some comments did not make sense in the context of the 
questions, 20 offered specific critiques borne out of experiences with assessments. 

We examined the 20 responses to assessment-related open question for shared key 
words and concepts, boldfacing the classifying phrases. For example, in their comments, some 
respondents focused on the design of the assessment device. These comments included phrases 
relating to the qualifications of the person creating the device, the device’s compatibility with 
the course, and the validity of externally required assessment. We coded those comments 
as design-focused. Other respondents’ comments focused on external entities using the data 
punitively or on the lack of useful data. 
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Respondent Demographics 

No faculty completing the survey were on the tenure track. The majority of respondents 
were teaching part-time (79%), while the rest held full-time positions with multi-year or 
tenure-like contracts (10%) or without these contracts (8%). Most of the respondents worked at 
either a 2-year institution (60%) or a 4-year institution (30%), although 10% of the respondents 
indicated they worked at both types of institution concurrently. Nearly 80% of respondents 
answered that they were teaching classes regularly, but only 39% felt that they had reasonable 
job security. Fifty-six percent of the respondents had been teaching for more than 11 years, 
while 29% had 4 to 10 years of experience and 8% marked that they had less than 4 years of 
experience. A majority of the survey respondents were 45 years of age or older (57%) and were 
female (53%). Only 14% of the respondents held a Ph.D., with 64% holding at least an M.A. and 
4% holding a J.D. Nearly half (48%) of respondents were hoping to find full time employment 
that emphasized teaching. (Percentages were rounded to closest whole number.) 

Limits to Interpretation 

Although the survey was sent to a large number of respondents, a selection bias effect 
is likely: Respondents were not compensated and the 35-question survey was estimated to 
take 10 to 15 minutes to complete. Thus, faculty choosing to respond to this survey may have 
been more likely to be interested in student learning outcomes (SLOs), assessment-aware, and 
teaching-oriented. We must also point out that the presence of faculty unions and legislative 
support in California has led to better working conditions for many of the faculty responding 
to this survey. 

In addition, the survey was conducted during fall term in 2012 when many community 
colleges in the region were filing College State Reports on SLO Implementation for the Western 
Association of Schools and Colleges’ (WASC) community college arm (2012), the Accrediting 
Commission for Colleges and Junior Colleges (ACCJC; Reminder, 2012). It is likely that this 
semester would result in more administration and committee communications with faculty, 
and thus increased formal and informal dialog about assessment. We must also note that the 
small n of the study limits the range of conclusions we can draw. 

Results 


Indications of a Divided Faculty 

One might argue that, in terms of assessment, faculty are faculty. Assessment measures 
student gains in knowledge and skills, which are related to teaching. Both NTT and tenure-line 
faculty are paid to teach as a part of their contracts, so one could also argue that both groups 
are paid to perform assessment as part of their classroom duties. However, such an argument 
takes a limited view of the process of assessing learning gains. Assessment devices and rubrics 
must be designed and implemented. The resulting data must be scored, analyzed, and applied. 
In fact, most of the assessment process happens outside of the classroom, which leads us to one 
of the potential differences between faculty. Tenure-track faculty generally work on a contract 
with a salary. The expectation is that the faculty member will teach, grade, hold office hours, 
perform departmental and institutional service, research, and advise. Assessment, arguably, 
falls under institutional and departmental service. 

In our previous study, we noted that the majority of NTT respondents reported they 
are not paid to do many things that are included in a standard contract for a tenure-line 
faculty member. For example, few are paid to attend department meetings (16.4%), attend 
training (13.4%), or hold office hours (34.3%). In addition, only 13.4% of NTT faculty reported 
being compensated for outcomes assessment (a figure we segregate from Table 1 because many 
tenure-line faculty also claim not to be compensated for such activities). These findings are 
consistent with the literature (Kezar, 2012; Patton, 2013; Street et ah, 2012) and conversations 
in academic forums, such as the Chronicle of Higher Education. While these findings are not 
surprising, they are important in establishing that faculty are not one homogenous group. It 
is logical to presume that a lack of pay might inhibit some NTT faculty from participating in 
assessment, but pay also has indirect effects. 


Relatively few respon¬ 
dents indicated they had 
heard about the true goal 
of assessment—improve¬ 
ment in teaching—either 
directly or indirectly. 


While 68% of respon¬ 
dents reported learning 
through direct channels 
that SLOs were required 
and 53% had heard 
through similar channels 
that SLOs had to be 
assessed in classes, only 
34.8% respondents had 
heard that their teach¬ 
ing methods should be 
updated based on the 
assessment findings. 
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Almost 57% of respon¬ 
dents were asked by their 
departments to assess 
classes, and of those, 
97.4% completed some 
form of assessment. 


Of these respondents, 
42.2% said they would 
assess more if they had 
a relationship with their 
department, or if they 
knew that participating 
would help them learn 
more about their teach¬ 
ing effectiveness. 


Pay causes a divide between faculty in several ways. First, it impacts what people 
choose or are able to do with their time—NTT faculty may teach at multiple campuses to make 
a livable income, leaving little time for unpaid meetings, office hours, or trainings. If faculty are 
not paid to be in office hours or in training sessions, they are unlikely to be spending additional 
hours in their departments or at department meetings (Chronicle Reporting, 2009). Thus, pay 
disparity creates a situation in which NTT faculty are less likely to have spare time to spend 
with colleagues or become informed on policy changes, especially those in the ever-changing 
world of assessment. These are important contract differences because they contribute to 
contingent faculty becoming detached from the decisions made by the department and the 
tenure track faculty in their disciplines, leading to two groups of faculty that are profoundly 
different from each other thanks to variance in institutional knowledge they possess. Second, 
department chairs may be unwilling to ask NTT faculty to participate in unpaid meetings 
or work, even if it relates to assessment, because the chair does not want to ask a person to 
work for free. Although the logic behind the exclusion is rational, in our survey, only 13% of 
faculty marked that they were paid for attending training, yet 64% said they still attended 
training sessions. The statistic is consistent with suggestions from Kezar and Sam (2011) 
that NTT faculty behave more like professionals than like hourly labor, with an interest in 
professional activity and development. These facts lead us to believe that additional uninvited 
NTT faculty would, if invited, also attend department meetings, and that even if they did not 
attend, the invitation might still help relieve anxieties by rendering the process more inclusive 
and transparent. 

Impacts of Direct and Indirect Messages 

Because we posit there are communication and institutional knowledge differences 
between the two faculty groups, we asked NTT faculty what they are hearing about assessment 
from varied sources. Hutchings (2010) and Kuh and Ikenberry (2009) have indicated that 
tenure-line faculty hear about accreditation and institutional plans for assessment from their 
institution’s administration. This information is distributed directly through institutional 
and departmental meetings, as well as indirectly through faculty discussions. NTT faculty, 
however, if not present at formal meetings, may receive the information indirectly, from 
alternate sources, or not at all. For this reason, we asked how our respondents heard 
about assessment and SLOs and what sorts of statements they heard. We classified the 
channels through which they obtained information as direct or indirect. Direct includes an 
announcement stated institutionally, from an administrator, department, or from an official 
meeting or workshop. Indirect information includes information obtained through informal 
discussion or a side conversation. 


Table 1 


Percentage of NTT Faculty Hearing the Following Expectations, by Direct or 
Indirect Channels 


Expectations 

Direct 

Indirect 

SLOs are required by accrediting agencies. 

68.2% 

71.2% 

SLOs promote increased student learning. 

50% 

30.3% 

SLOs have to be tied to material from class to 
demonstrate where students learn. 

48.5% 

42.4% 

SLOs have to be assessed in each class to show what 
students are accomplishing. 

53% 

47% 

Faculty need to show how they updated their teaching 
methods in accordance with the results of the 

34.8% 

33.3% 

assessments. 



SLOs and assessment are likely to disappear. 

0% 

1.5% 


N=66*; 1 respondent skipped these questions on the survey 
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were aware teaching methods should be updated based on results. Taken together, participant 
responses suggest that reasons for collecting assessment data are not being as heavily stressed 
as the vague concept of assessment is. 

Implications. We can only speculate about what is happening to cause the above 
imbalance, but the findings support the statement that messages are not conveyed consistently. 
Many of our respondents’ departments may not have been discussing the results of previous 
assessments ratings or scorings with NTT faculty. If departments had contacted NTT faculty 
about the results of previous assessment attempts, perhaps through the scheduling of teaching 
workshops or special meetings to discuss ways to increase student learning and interaction 
in classes, we expect that more respondents would have stated that they had heard about 
the need to update their teaching. Or perhaps, if departments or institutions were trying 
to inform NTT faculty about a discipline’s assessment project’s relationship to teaching 
methods, they were not doing so in a way that was heard or recognized. For instance, such 
communications may have been treated as formalities, to be done once and then checked 
off as completed. Communications that are not “on message” and not repeated often are 
likely to be missed or misconstrued. By contrast, communications about accreditation 
may have been more direct, more on-message, and more frequently repeated due to the 
immediate anxieties of administrations. Regardless, one can expect NTT faculty concerns 
about assessment to increase when instructors are unaware how the process is about helping 
faculty improve their teaching. 

Involvement in Assessment Activities 

Considering the push by accrediting institutions to increase the number of classes 
and sections assessing SLOs, we asked our respondents whether their departments were 
implementing assessment processes in their courses without their involvement or were 
asking them to assess SLOs in their sections. Almost 57% of respondents were asked by their 
departments to assess classes, and of those, 97.4% completed some form of assessment. Although 
those numbers sound encouraging, 30.8% reported their department or discipline administered 
an assessment in a section without the faculty member’s involvement. If NTT faculty are not 
involved in the design, discussion, or administration of department-run assessments (which, 
unlike institutional tools, tenure-track faculty often influence), then the results may be less 
useful for them. If they are not given the results culled from the instruments, then they are left 
with less information to update teaching methods. 

Predicted Effects for Incentives to Assess 

As a part of the survey, we listed potential scenarios that might increase participation 
in the creation, implementation, and ratings of assessment instruments. Not surprisingly, a paid 
stipend was picked the most often (60.9%), with one respondent writing that compensation 
for assessment should be explicitly included in the instructor’s contract. Another echoed, 
“[T]he college has given so little guidance to part-time faculty about specific SLOs and their 
development. We’ve been asked to voluntarily develop them because of the accreditation 
process but no compensation is available.” 

Nevertheless, many survey respondents showed an interest in participating for reasons 
other than money. Of these respondents, 42.2% said they would assess more if they had a 
relationship with their department, or if they knew that participating would help them learn 
more about their teaching effectiveness. These findings echo those of earlier studies showing 
that NTT faculty are very concerned with being strong and effective teachers (Baldwin & 
Wawrzynski, 2011; Kezar & Sam, 2011; Levin & Shaker, 2011; Scott & Danley-Scott, in press). 
These numbers are summed up in the words of one respondent: “I will do it because I think it’s 
important; [I] don’t need further incentives.” 

Patterns in Open Comment Responses 

When we look at the written comments of the NTT faculty to determine why they 
would feel uncomfortable with departments administering SLO assessments in classes, we 
see three strong trends. Respondents expressed concerns about usefulness of collected data, 
control of assessment, and punitive actions based on data. 
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Two of the comments 
express frustration that 
the faculty or adminis¬ 
trators designing their 
departmental assess¬ 
ments were not familiar 
with the material and 
the pedagogy used in 
the classroom, possibly 
leading to misleading or 
misinterpreted findings. 


Given that NTT faculty 
employment is tenuous 
and unprotected, it 
is not surprising that 
some believe assess¬ 
ment results could be 
used against them or 
inappropriately. 
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It is clear from the first set of comments that NTT faculty concerns over whether 
assessment data are useful, are consistent with the general “faculty” perspectives described 
by Fort (2011), Hutchings (2010), and Lederman (2010b). The respondents question whether 
assessment instruments accurately measure student learning in an instructor’s class. If students 
do not understand the questions or material presented in the assessment, their progress may 
be inaccurately measured. Similarly, if students are not given a good reason to take assessment 
activities seriously or if the activities do not take into consideration course format (online, 
hybrid, evening), data may be misleading. This concern is most likely to exist when those in 
the classroom are not involved in designing the assessment instruments or determining the 
learning outcomes. While the view is not entirely unique to NTT faculty, exclusion from the 
assessment process may heighten these concerns beyond those of tenure-line faculty. 

Within the second set of comments, some responding faculty remarked that 
assessment instruments and activities created by others (faculty or administrators) may not 
be appropriate for the courses they are teaching. Two of the comments express frustration 
that the faculty or administrators designing their departmental assessments were not familiar 
with the material and the pedagogy used in the classroom, possibly leading to misleading or 

Table 2 

Open Responses: Why Would You FeeI Uncomfortable Having Your Department Administer SLO Assessments in 
Your Class? Which of the Following Would Make You More Likely to Participate in Assessing the SLOs in Your 
Class? (“Other" Open Responses) _ 

Concern 1: Lack of useful data 

-The uncomfortableness comes from being told to put them in the syllabi, knowing some may not be tailored to 
your class specifically, and then hoping the students are understanding or at least comprehending most of them, 
if not at least knowing they are supposed to be? 

-Students may perform differently if they sense someone other than their teacher will be evaluating them. 

-I would need to be convinced [the results] are useful. 

-Continuity about what is deemed important for students to know after completing the course. 

Concern 2: Control over assessment design and procedure 

-It’s my class and it would be my responsibility. 

-Prefer to do it myself. 

-No control over the assessment, the process or outcome. 

-The department assures me that they don't have the quals to teach the class and they love the results from the 
students who have taken it. I'm not confident that the faculty actually understands the domain well enough to 
do this without me. 

-It takes away from my own authority in class. 

-It's not having my class assessed— I do that myself. I would have a problem having someone else do it. I am the 
one who has taught the material based on the putpose of the class and the looked for terminal objectives. 

-Because the important SLOs are complex - e.g. critical thinking - and I find the best assessment strategy is to 
build it into an assignment which I can then improve in subsequent semesters based on assessment results. 

-I know more about teaching writing than anyone in my department. The only tenured faculty member in the 
university writing program is an administrator, not a writing teacher. He is not qualified to assess my class, in 
my opinion. In fact, he asks me for advice about certain issues. 

-Instructions on what exactly I need to do. 

-I believe that the pursuit of SLO's is a noble goal, but is misplaced at the collegiate level. The emphasis of a 
college education shoidd be to expose students to new concepts and idea's while trusting that it is the students 
responsibility to learn the material. I feel that SLO assessments only sen’e to corrode the college learning 
environment with an ultimate result of making the completion of a four year degree as formatted and formulaic 
as secondary education has become. The responsibility for actually learning the course material must always 
reside with the student and not the educator. 

-If assessed as a group, the most important factor is the day/lime of the meeting. 

-Because the college has given so little guidance to part-time faculty about specific SLOs and their development. 
We've been asked to voluntarily develop them because of the accreditation process but no compensation is 
available. 

Concern 3: Use of assessment data 

-Administrators often have a way of making assessments on student progress about ability of an instructor to 
teach the concepts. Administrators heed no regard to other criteria such as student participation, interest or 
application of the concepts learned. Further, assessments of SLO's are of no interest to students and they may 
choose to skew their answers. 

-I do not trust the judgment of administration. 

-I don't trust them. It can be used against me. 

-I would need more information. Without it I would feel scrutinized. _ 
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misinterpreted findings. 

Apprehension about the alignment of assessment instruments with course content 
may be particularly rational for those off the tenure line. If NTT faculty are not involved in 
their departments, then the assessment instruments are likely to be designed by administrators 
or tenure-line department faculty. NTT faculty are most likely to teach the introductory 
courses and tenure-line faculty often teach the upper-division courses at four-year institutions 
(Benjamin, 2002), so there may be a disconnect among expectations of what the students 
should be able to accomplish or what is assigned in class. Moreover, because NTT faculty are 
often hired at the last minute (as noted earlier), they are unlikely to communicate with other 
faculty in the department until after the semester starts. Tenure-line faculty are more likely to 
have developed their syllabi and course content toward a departmental norm over the years, 
due to discussions with colleagues, performance reviews, and mentoring. Combining these 
facts with the tradition that instructors have freedom over topics, assignments, and materials 
presented in class, the syllabi and course content may vary significantly between tenure-line 
and NTT faculty. If assessments are designed based on the norms of tenure-line faculty classes, 
there may be incorrect assumptions about which books, assignments, and activities occur in 
NTT-taught classes. Student learning may occur in the latter classes, but if that progress fails 
to appear in the same topics or skills, or does not align with departmental norms, the data will 
not show a growth in mastery. 

Third and finally, some respondents voiced concern over how the assessment data will 
be used. Given that NTT faculty employment is tenuous and unprotected, it is not surprising 
that some believe assessment results could be used against them or inappropriately. When 
departments and institutions do not share how the assessment device will be constructed, 
which outcomes will be measured, and how the resulting data will be used, the faculty are 
left to assume the data will be used for evaluation or judgment. Although tenure-line faculty 
do express concern over how assessment data is used (Gold et al., 2011; Hutchings, 2010; 
Lederman 2010b), they attend departmental and division meetings that discuss assessment, 
where they can opt to participate in the process. In comparison, NTT faculty are often in the 
dark and cannot choose to participate more than they do. The final category of comments, 
those that discuss concerns with the use of assessment data, may be the easiest for institutions 
to alleviate. If data are used in workshops, newsletters, or departmental memos on improving 
pedagogy or teaching techniques, the concerns may decrease over time, assuming the 
department ensures the NTT faculty are made aware of these events and communications. 

Differences Among Those Making Comments 

It is worth noting that the respondents in Set 3—those who expressed concerns over 
how assessment data might be used—shared a teaching profile strikingly different from that 
of the other respondents, coming closest perhaps to respondents within Set 1, but differing 
markedly from those in Set 2 and from those not leaving comments. 

All Set 3 respondents at the time of the survey taught part-time for two-year colleges 
and had taught for more than seven years. None had a doctorate. None indicated having 
received any department assistance with assessment design, none had faculty mentors, and 
none were paid to conduct assessments. Just one in four felt the department treated him or 
her well, only one in four felt his or her job was secure, and just one in four was invited to 
department meetings. 

The foregoing demographic contrasts sharply with respondents who gave no response 
when asked what about the assessment response made them uncomfortable. Although lack of 
response might include respondents in a rush to finish, it also by definition includes respondents 
who had no concerns about assessment in their programs. The no-comment group featured 
more respondents with doctorates (21%), as well as respondents with better-than-normal job 
security: 23% of non-commenters had full-time jobs or a secure employment contract, and 45% 
indicated they felt secure in their jobs. The non-commenting population had also indicated 
through earlier answers that they were more involved in their departments: 19% had faculty 
mentors, 57% had been invited to department meetings (indeed, 17% indicated they were 
compensated for such meetings). They had also indicated more support for assessment efforts, 
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with 40% reported having received department assistance with assessment and 17% indicating 
they had been paid for assessment activities. 

Respondents expressing concern about punitive actions also differed sharply from 
those making Set 2 comments—those expressing concern about assessment design and 
procedure. In fact, the respondents who focused on design and process questions seemed 
much more like the non-commenters than like others who had commented: 17% had full¬ 
time positions or secure contracts, 33% felt they had reasonable job security, 67% were 
invited to department meetings (with 25% paid to do so), 17% had mentors, 8.3% were paid 
to assess, and as with the non-commenters, 42% reported receiving department assistance 
with assessment design. 

In short, respondents who felt secure in their jobs and whose other responses indicated 
more opportunities for department involvement seemed more likely to focus on the how of 
assessment, if they commented at all, while their more sidelined peers tended to focus on the 
why. Most units with any experience with administering assessments are likely to find the how 
discussions more productive than the why discussions, and judging from the data above, it 
seems like one way to shift discussion from why to how might be to give NTT faculty a wider 
range of ways to participate. 


Conclusion 

Our findings suggest departments that want to have increased NTT involvement in 
distributing, collecting, and rating assessments need to increase opportunities for involvement 
in assessment design, and they need to open communication between the institution and 
the NTT faculty. To further reduce anxiety over assessment, departments should close the 
loop with their assessment data and show the pedagogical uses for data. When departments 
use assessment data the way it was meant to be used, trust in the department’s practice of 
assessment should increase. Increased participation in assessment should, in turn, lead to 
more awareness of what practices improve student mastery of student learning outcomes. 

Our results support scholars who argue that institutional factors, rather than faculty, are 
leading to a less-effective teaching environment. Other studies have highlighted the frustrations 
that assessment is an unfunded mandate (Funk & Klomparens, 2006; Gilbert, 2010; Gold et al., 
2011), that it is not recognized appropriately for the energy and time it requires (Hutchings, 
2010), and that it is “making relatively little difference on their campuses” (Lederman, 2010b, 
para. 10). The previous literature shares a presumption that all faculty have these concerns 
with assessment, but NTT faculty we surveyed have additional concerns that may be of higher 
priority to them, due to a different job description, less-inclusive compensation package, and 
departmental norms on communication and involvement. 

NTT faculty are concerned with (a) whether assessment data are useful; (b) whether 
the findings from assessment attempts will affect their tenuous employment; and (c) whether 
the assessment instruments will properly measure learning gains in their classes. If we 
look at these three concerns, a clear picture emerges: When departments and institutions 
do not involve their NTT faculty in the design, implementation, and analysis of assessment, 
particularly in the assessment of general education courses, NTT faculty insecurities will rise 
and participation in assessment implementation and ratings will likely decrease. Given only 
39% of our sample felt they had reasonable job security, the respondents would be rational to 
distrust administrators collecting random artifacts and data from classes. 

As accrediting bodies push institutions to perform discipline- and department-level 
assessment of student learning outcomes, it will be increasingly common to see such units 
assessing all sections of a class or major using a common assessment device and a common 
rubric. This practice leads to centralized assessments and centralized scoring, as well as a 
collection of data about individual instructor’s courses and student success. Even if it seems 
like a small intervention, units should involve NTT faculty in assessment efforts. Nearly half 
of our respondents reported that they would be involved in the assessment process if it helped 
them learn about the effectiveness of their teaching. Given NTT faculty teach a large percentage 
of introductory classes and are interested in the pedagogical benefits of measuring student 
learning gains, assessment data have a higher likelihood of producing change if departments 



ensure usable data is distributed to all faculty. Developing instruments and analyzing results 
in the dark may encourage passive resistance toward assessment and resentment toward the 
department, both of which may taint the results or decrease participation. 

In addition, communication with this half of the faculty body needs to improve. One 
should assume that faculty left out of department and assessment meetings hear only bits and 
pieces of announcements and news, or that the informants may frame plans and policies based 
upon personal feelings and concerns. Departments might believe important details will filter 
down the ranks, but as only 16.9% of respondents stated they had some type of mentoring 
relationship with a full-time faculty member, it is unlikely accurate information will reach a 
majority of the contingent faculty. Formal departmental communication, via meetings, emails, 
or memos, would aid in conveying essential information accurately and building a relationship. 

Lack of personal communication between NTT faculty and permanent members of an 
institution, such as administrators, departments, and tenure-line faculty, is also contributing 
to the problem of participation in student learning outcomes assessment. Policies that make 
it difficult for NTT faculty to be a meaningful part of a department will also affect whether 
all faculty feel allowed to offer suggestions about assessment design and implementation. 
Overlooking those teaching the classes being evaluated can also reduce the accuracy of 
measuring student learning gains. It reduces teaching effectiveness because the data is not 
optimal and may be ignored. As communication and openness are inexpensive ways to reduce 
concern, and have a great impact on the desire to participate in assessment (Scott & Danley- 
Scott, in press), we again recommend opening dialog and encouraging participation to increase 
participation and create useful data that can increase teaching effectiveness. 
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